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COMPUTER  SIMULATION  OF  A  SPACE-BASED 
INFRARED  SURVEILLANCE  SENSOR 


1.  INTRODUCTION 

Simulation  of  a  Naval  Research  Laboratory  space-based  infrared  surveillance  sensor  must  include  such 
basic  effects  as  the  time-varying  geometry  of  the  satellite-to-earth  line  of  sight  (LOS),  atmospheric 
emission  and  attenuation,  the  optical  blur  of  the  sensor’s  telescope,  and  the  noise  and  nonuniform 
response  characteristics  of  the  solid-state  detectors.  A  user-oriented,  menu-driven  software  package  called 
the  End-to-End  Simulation  Package  (EESP)  has  been  developed  to  peiform  the  necessary  calculations. 
In  addition  to  generating  accurately  simulated  imagery,  it  includes  a  set  of  signal  processing  algorithms 
for  finding  moving  targets.  The  EESP  computation  begins  with  real  IR  data  (TIMS  |1)  and  Hi-Camp  [2] 
are  the  primary  sources),  accepts  user  inputs  for  many  parameters  (e.g.,  orbital  altitude  and  position  of 
satellite,  pixel  size,  ground  and  observation  point),  and  ends  with  target  declarations.  In  between,  all 
effects  must  be  taken  into  account  with  sufficient  accuracy  to  give  valid  results. 

Tlie  simulated  seii.soi  is  a  satdlite-burne,  scanning,  IR  radiometer.  The  satellite  is  placed  in  a  circular 
orbit,  unrestricted  in  altitude.  Figure  1  shows  the  basic  geometry  of  the  problem,  llie  linear  focal  plane 
array  is  mounted  in  a  telescope  which,  at  the  user’s  option,  can  either  be  placed  on  a  gimbal  or  use  an 
external  steering  mirror  to  achieve  LOS  pointing  control  (3].  The  linear  focal  plane  array  consists  of  512 
detectors  arranged  “vertically”  and  scans  "horizontally”  across  the  scene  through  512  pixels 
(pixel  ®  resolution  element)  by  means  of  a  scan  mirror.  Vertical  spacing  is  under  user  control  (.so  the 
detectors  can  be  overlapped),  as  is  the  sampling  rate  in  the  horizontal  (scan)  direction. 

EESP  has  demonstrated  the  capability  of  reducing  background  clutter  to  the  sensor  noise  level  for 
highly  structured  scenes.  Unfortunately,  this  claim  cannot  be  made  for  scenes  that  contain  clouds,  and 
hence  exhibit  cloud  parallax  motion,  but  EESP  has  the  capability  of  studying  this  problem  by  implanting 
moving  clouds  in  a  scene.  Preliminary  investigation  indicates  that,  while  clutter  due  to  cloud  parallax 
motion  cannot  be  processed  out  to  the  level  of  sensor  noise,  it  can  be  drastically  reduced. 

2.  THE  END-TO-END  SIMULATION  PACKAGE  (EESP) 

In  this  section,  the  calculations  performed  by  EESP  to  generate  images  and  detect  targets  are  broken 
down  into  twelve  categories  and/or  capabilities.  Most  of  these  "items”  (as  they  will  be  referred  to 
hereafter)  require  further  explication.  This  is  done  partly  in  later  sections  of  this  report  and  partly  in  other 
reports  which  are  referred  to  as  needed.  The  flow  chart  provided  in  Fig.  2  shows  tlie  functional  relations 
of  the  calculational  items  briefly  described  below.  A  few  other  items,  which  are  on  the  EESP  menus  for 
the  purpose  of  doing  image  diagnostics,  are  presented  in  Section  8. 
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Fig.  2  —  Flow  chart  of  the  calculations  performed  by  EESP 
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2.1  Item  (1)  -  Read  Background  Data 

Real  infrared  data  taken  in  nadir-looking  mode  from  airborne  platforms  are  used  as  ground  truth. 
TIMS  and  Hi-Camp  are  our  major  sources.  Tlie  main  requirement  on  this  data,  other  than  that  it  be  of 
good  quality,  is  that  it  have  substantially  better  resolution  than  that  of  the  simulated  sensor.  The  10-m 
resolution  of  TIMS  and  Hi-Camp  fulfills  this  condition  with  respect  to  the  50  to  100  m  resolution  we 
believe  necessary  for  the  space  sensor.  Test-pattern  scenes,  such  as  checkerboards  and  intensities  that  are 
linear  functions  of  position  are  also  available.  Hereafter,  this  background  data  will  be  referred  to  as  HR 
(high  resolution)  data. 

2.2  Item  (2)  ■  Implant  Targets 

Targets  can  be  inserted  into  the  HR  data,  -vith  user-specified  altitude,  flight  direction,  and  speed. 
Targets  are  modeled  by  a  compact  configuration  of  HR  pixels  with  an  appropriate  area.  Thus,  for 
example,  a  Backfire  bomber,  which  has  an  area  (vi**wed  from  above)  of  310  m\  is  modeled  by  3  HR 
pixels  (100  m^  per  pixel)  with  unit  emissivity  and  a  specified  radiance.  Since  the  target  is  unresolved,  a 
detailed  model  of  its  shape  is  unnecessary. 

2.3  Item  (3)  -  Implant  Clouds 

Very  little  data  exists  that  can  be  used  to  obtain  realistic  cloud  parallax  motions.  Therefore,  software 
has  been  written  so  that  artificial  clouds  can  be  implanted  in  the  scene.  These  clouds  can  be  assigned 
speeds  appropriate  to  their  altitudes  so  that  they  move  acro.ss  the  scene.  They  emit  their  own  radiation 
and  transmit  background  radiation  according  to  a  user-assigned  opacity,  which  can  vary  from  zero  to 
unity. 

2.4  Item  (4)  -  Atmospheric  Correction 

Atmospheric  transmission  and  path  radiance  for  arbitrary  zenith  angles  are  found  for  the  appropriate 
waveband  by  using  tables  of  these  quantities  at  particular  off-zenith  angles  (multiples  of  10°),  for 
appropriate  patlis  to  space  (surface-to-space  or  cloud-to-space),  then  intei-polaling  as  needed.  The  tables 
must  be  calculated  separately  (with  LOWTRAN  6,  for  example)  and  entered  into  the  program.  The 
correction  equation  is 

D'  =  (D  -  R)*T'/T+R' 

where 

D,  D'  are  the  original  and  corrected  data,  respectively, 

R,  R'  are  the  atmospheric  path  radiances  for  the  aircraft  and  for  the  satellite,  respectively,  and 
T,  T'  are  the  atmospheric  transmittances  for  the  aircraft  and  for  the  satellite,  respectively. 

Thus,  the  effect  of  the  atmosphere  is  first  removed  from  the  data  for  the  aircraft  altitude  and  viewing 
geometry,  then  reinserted  for  the  altitude  and  viewing  geometry  of  the  satellite. 

2.5  Item  (5)  -  LOS  Pointing  and  Tracking  (see  Section  3) 

Double-precision  ray-tracing  calculations  are  performed  to  locate  the  ground  footprints  of  the  array’s 
detectors  accurately  on  the  Earth’s  surface,  accounting  for  the  time-dependent  geometric  effects  of  die 
satellite  orientation  and  pointing  mechanism.  Either  a  gimbaled  telescope  or  an  external  steering  mirror 
can  be  used  for  LOS  pointing;  both  are  dealt  with  in  considerable  detail  in  Section  3.  Random  LOS  jitter 
can  be  added  as  a  user-specified  rms  value  (in  microradiuns).  At  this  point,  only  the  locations  of  the 
corner.^  of  “blocks”  (that  will  later  be  divided  into  64  x  64  pixels)  are  calculated  (see  Section  4).  They 
are  given  in  (latitude,  longitude)  coordinates. 
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2.6  Item  (6)  -  Convert  Latitude-Longitude  to  HR  Pixels  (see  Section  4) 

Block  footprint  points  on  the  Earth’s  surface  are  converted  from  latitude  and  longimde  to  HR  pixel 
coordinatCvS.  For  this  purpose,  the  HR  scene  is  “pasted”  onto  the  Earth  with  its  lower  left-hand  comer 
at  the  center  of  the  fie'd  of  view  (FOV)  (this  is  explained  in  detail  in  Section  6).  Once  HR  coordinates 
are  determined,  each  block  is  divided  into  4096  (64x64)  pixels. 

2.7  Item  (7)  -  Optical  Blur  Function  (see  Section  5) 

Optical  blur  is  simulated  by  convolving  the  telescope’s  blur  function  (taken  to  be  a  Gaussian)  with 
the  sensitive  area  of  a  square  detector.  The  blurring  that  results  from  the  scan  motion  is  included  by 
integrating  over  the  positions  taken  by  the  blur  spot  as  the  scene  is  scaimed  across  the  focal  plane.  The 
final  blur  function  (which,  because  of  the  scanning  motion,  is  different  in  the  x-  and  y-directions)  is 
stored  in  a  look-up  table  for  later  use. 

2.8  Item  (8)  -  Integrate  over  Background  Scene  (see  Section  6) 

For  each  sample  in  the  simulated  scene,  the  HR  pixels  that  make  a  significant  contribution  to  a 
simulated  pixel  must  be  located,  and  their  contributions  to  the  sample  calculated,  as  described  in  detail 
in  Section  6.  Location  of  the  relevant  HR  pixels  is  accomplished  by  the  “folding  in”  process  described 
in  Section  6. 1 . 

2.9  Item  (9)  -  Add  Noise 

Detector  noise,  considered  to  be  white,  is  added  to  the  simulated  data  using  Gaussian-distributed 
random  numbers  with  standard  deviations  specified  by  the  user. 

2.10  Item  (10)  -  Add  Nonuniformity 

Detector-to-detector  calibration  nonuniformily  is  added  to  the  simulated  data  using 
Gaussian-distributed  random  numbers  with  standard  deviations  specified  by  the  user.  Since  a  scanning 
sensor  is  assumed,  this  means  that  each  horizontal  row  of  samples  in  the  simulated  imagery  comes  from 
a  detector  with  different  responsivity  compared  to  its  neighbors.  This  results,  of  course,  in  a  “streaky” 
picture  [4]. 

At  this  point,  a  complete  simulated  picture  exists.  The  example  shown  in  Fig.  3  was  obtained  from 
far-lR  imagery  of  the  countryside  near  Mono  Lake,  CA.  A  256 x 256-sample  simulated  image  was 
generated  for  an  assumed  satellite  altitude  of  800  km,  with  50  m  ground  resolution  at  nadir.  However, 
the  scene  shown  used  far  off-nadir  viewing  geometry  (grazing  angle  =  30°),  resulting  in  a  ground 
resolution  of  87  m  x  175  m  due  to  the  greater  range  to  the  tracked  ground  point  and  to  foreshonening 
of  the  scene.  I’here  are  two  samples  per  pixel  in  this  image;  the  sampling  rate  is  a  user  specified  input 
and  is  explained  in  detail  in  Section  4.  Up  to  20  pictures  can  be  generated  (a  larger  number  may  require 
some  code  modifications)  to  obtain  a  time-sequential  string  of  images  for  testing  signal  processing 
algorithms.  The  major  algorithms  in  the  software  package  are  listed  in  Sections  2.1 1  and  2.12. 

2.11  Hem  (11)  -  Signal  Processing  (see  Section  7) 

Signal  processing  techniques  for  rendering  faint  moving  targets  detectable  can  be  applied  in  any  order 
or  not  at  all,  at  the  user’s  option.  More  algorithms  can  be  added  in  a  modular  fashion. 
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Fig.  3  —  A  simulated  far-lR  image  of  the  desert  near  Mono  Lake, 
CA.  The  hori7X)ntal  fold  point  just  above  the  center  is  explained  in 
Section  3  (cf.,  Fig.  10). 


2.11.1  Registration  ami  Resampling 

Successive  pictures  are  resampled  on  a  common  grid.  This  requires  finding  the  displacement  between 
them  [4,5]  and  applying  some  sort  of  interpolator  (6,7],  and  is  intended  primarily  to  prepare  the  data  for 
the  Frame  Differencing  Signal  Processor  (FDSP).  A  few  additional  remarks  on  registration  and 
resampling  are  made  in  Section  7. 

2.11.2  Frame  Differencing 

Subtracting  frames  of  data  can  be  done  to  any  desired  order.  Stationary  backgrounds  can  thus  be 
subtracted  out  (with  a  precision  limited  by  the  registration  and  resampling  algorithms  indicated  above, 
and  by  inherent  scene  distortion)  and  aircraft  in  flight  detected  by  their  motion:  with  50  to  100  m 
resolution,  they  are  too  faint  to  find  any  other  way.  Experience  indicates  tfiat,  for  good  registration  and 
resampling  algorithms,  first  order  differencing  is  sufficient;  higher  orders  are  completely  dominated  by 
detector  noise  and  nonuniformity.  In  the  presence  of  cloud  parallax  motion,  however,  second-order 
differencing  appears  to  be  useful. 

Figure  4  shows  a  difference  frame  generated  for  the  scene  shown  in  Fig.  3,  using  two  images 
separated  in  time  by  0.5  s.  The  dark-bright  spots  (“target  dipole”)  at  the  center  of  the  picture  are  the 
signature  of  a  faint  moving  target  (a  Backfire  bomber),  completely  invisible  in  Fig.  3. 

2.1 1.3  Spatial  Filters 

Currently,  only  a  median-subtract  spatial  filter  has  been  implemented.  Spatial  filters  have  very  little 
effect  on  the  output  of  a  frame-differencer  if  scenes  can  be  registered  well  before  differencing;  their 
primary  use  is  for  larger,  slower,  targets  than  aircraft  (e.g.,  ships). 
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Fig  4  —  The  difference  between  the  image  of  Fig  3  and  another 
image  taken  30  s  later.  The  dipole  signature  of  a  moving  target  is 
apparent  at  the  center  of  the  scene.  Note  other  largeliike  features  in 
the  image. 


2.12  Item  (12)  -  Track  Assembly 

The  signature  of  a  moving  target  in  a  frame  differencing  signal  processor  is  a  “dipole,"  a  positive 
contrast  feature  adjacent  to  a  negative  contrast  feature.  The  direction  and  magnitude  of  the  feature’s 
separation  is  determined  by  the  velocity  of  the  target.  The  tracking  algorithm,  called  die  logical  tracker 
[8],  searches  the  first  difference  frame  for  all  dipoles  that  correspond  to  realistic  veiocnies.  then  checks 
to  see  if  similar  dipoles  appear  at  the  places  appropriate  to  their  velocities  :n  subsequent  difference 
frames. 


Figure  5  shows  die  result  of  applying  the  logical  tracker  to  a  sequence  of  difference  frames  of  which 
Fig.  3  is  the  first.  The  mo  Ing  target  is  detected  and  there  are  no  false  alarms,  even  though  Fig.  4  shows 
a  number  of  point-target-1  ikt  features . 

The  items  above  requiring  fuither  explanation  arc  covered  in  greater  depth  in  the  following  sections. 

3.  LINE-OF-SIGIIT  POINTING  AND  TRACKING  (ITEM  5) 

The  s.atellite  is  placed  in  a  northbound,  inclintxl,  circular  orbit,  at  die  initial  (latitude,  longitude)  point 
of  (0,0)  at  t  ==  0.  Thus,  if  the  user  cliuo.ses  an  orbital  inclination  of  90°,  the  satellite  is  placed  in  a  polar 
orbit,  northbound  on  the  Greenwich  Meridian,  directly  above  the  equator  at  t  =  0  (refer  to  Fig.  1,  which 
does  not,  however,  show  die  satellite  on  the  Greenwich  Meridian).  A  different  orbital  position  can  he 
obtained  by  .specifying  t  0. 
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Fig.  5  —  The  target  track  obtamed  from  Fig.  4  and  subset^uent 
differenced  images.  The  largetlike  clutter  features  of  Fig.  4  have 
been  rejected  on  'he  basis  of  their  (lack  of)  motion. 


The  linear  focal  plane  array  is  mounted  in  a  telescope  which,  at  the  user’s  option,  can  either  be 
placed  on  a  gimbal,  as  shown  in  Fig.  6,  or  use  a  fast  steering  mirror  (FSM),  external  to  the  telescope, 
to  achieve  LOS  pointing  control.  Except  for  its  optical  blur  function,  the  actual  properties  of  the  telescope 
are  not  used  in  the  calculation.  The  telescope  serves  to  convert  linear  measures  in  the  focal  plane  to 
angular  measures  in  the  FOV;  the  calculation  simply  assumes  the  angular  measures  with  which  to  begin 
(mainly  this  means  the  angular  IFOV  of  the  detectors),  and  proceeds  from  there. 

flight  direction 


OUTER  j  , 

gimbal  " ,  /  itD 

AXIS  !  / 

1 


f 

LOS  TO  GROUND 

Fig.  6  —  The  lu.o-axis  gimbal  moves  Ihc  sensor  package  to  track  the 
desired  ground  p>oint.  When  the  pitch,  roll,  and  yaw  of  the  satellite 
arc  zero,  the  outer  gimbal  axis  (G2)  is  parallel  to  the  flight  direction. 
Scanning  is  done  with  a  mirror  internal  to  the  sensor  package. 
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The  linear  focal  plane  array  (FPA)  consists  of  512  pixels  (pixel  =  resolution  el ement= detector  size) 
that  are  arranged  "vertically”  and  scanned  “horizontally”  across  the  scene  through  512  pixels  by  means 
of  a  scan  mirror.  Thus,  the  "horizontal”  and  “vertical”  directions  are  the  scan  and  cross-scan  directions, 
respectively.  The  steering  mirror,  when  used,  is  separate  from  the  scan  mirror.  The  FPA,  telescope,  and 
scan  mirror  assembly  are  shown  schematically  in  Fig.  7.  Comparison  of  Figs.  7  and  1  will  show  the 
rationale  behind  the  “horizontal”  and  “vertical”  terminology. 


x4 

SCAN  MIRROR 


LOS  TO  GROUND 


I 


Fig.  7  —  The  scan  mirror  rotates  about  the  Z4  axis 
(which  is  fixed  in  the  saiellitc)  to  scan  the  LOS  from  left 
to  right.  When  the  pilch,  roll,  and  yaw  of  the  satellite  arc 
zero,  Z4  is  in  the  flight  direction. 


The  512  X  5 12  image  is  the  largest  the  simulator  can  generate.  In  fact,  the  usual  practice  is  to  generate 
only  a  64(vert)x256(hor)  image.  One  reason  for  this  is  to  save  computing  time,  because  it  can  take  up 
to  30  min  to  generate  even  this  small  image  when  the  calculation  is  done  for  near-horizon  viewing 
geometry  (near-nadir  viewing  takes  only  5  to  10  min).  The  other  reason  is  that  the  background  data  used 
covers  only  a  small  part  of  the  footprint  of  a  satellite-borne  sensor. 

The  sensor’s  FOV  is  determined  by  the  length  of  the  array  and  the  amplitude  of  the  scan  mirror’s 
motion;  it  is  512x512  pixels.  A  careful  distinction  must  be  made  between  pixels,  which  are  determined 
by  the  size  of  detectors,  and  samples,  which  are  determined  by  how  often  a  detector’s  output  is  sampled 
(in  the  scan  direction)  or  what  the  center-to-center  spacing  of  the  detectors  is  (in  the  cross-scan  direction). 

Physically,  of  course,  the  spacing  of  samples  in  the  vertical  direction  must  be  accomplished  with  some 
staggered  layout  of  the  detectors  in  the  FPA,  but  this  is  not  something  the  computer  has  to  worry  about. 

The  program  can  handle  more  than  one  sample  per  pixel  (but  not  less!). 

If  one  sample  per  pixel  is  specified,  then  the  final  64(vert) x 256(hor)  image  will  cover  l/16th  of  the 
sensor’s  FOV.  If  more  than  one  sample  per  pixel  is  specified,  then  the  64  x 256-sample  image  will  cover 
a  smaller  fraction  of  the  total  FOV.  The  64  x  256-sample  image  is  chosen  so  that  its  right-hand  edge 
coincides  with  the  centerline  of  the  512x512  pixel  FOV.  ^ 

To  solve  the  pointing  and  tracking  problen.  and  to  calculate  ground  footprints,  a  number  of 
coordinate  sets  and  the  angular  transformation.<^  bet  ten  them  will  be  needed.  Four  sets  of  coordinates  < 

are  used. 
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(1)  (xl,yl,zl)  are  Earth  coordinates,  shown  in  Fig.l.  In  terms  of  radius  R,  latitude  d  (nortfi  latitude 
is  positive),  and  longitude  4*  (east  longitude  is  positive),  they  arc 

xl  =  R  cosd  coS(i», 

yl  =  R  cos^  COS0, 
and 

zl  =  R  sin^. 

For  points  on  the  Earth’s  surface,  R  =  R,  -  Earth  radius. 

(2)  (x2,y2.z2)  are  orbiting  coordinates,  shown  in  Fig.  2.  Coordinate  x2  is  the  local  vertical,  z2  is  the 
direction  of  flight.  These  axes  are  assumed  to  rotate  so  as  to  track  the  local  vertical  as  the  satellite  orbits. 

(3)  Coordinates  (x3.y3,z3)  are  satellite  coordinates.  They  are  related  to  (x2,y2,z2)  by  the  roll,  pitch, 
yaw  (R,P,Y)  orientation  angles  of  the  satellite,  which  are  zero  in  nominal  orientation.  For  the  gimbaled 
telescope,  LOS  pointing  is  done  entirely  with  the  gimbaJ;  R,P,Y  remain  zero.  When  the  FSM  is  used, 
the  satellite  is  maneuvered  in  pitcli  and  roll  to  obtain  initial  orientation  (see  below). 

(4)  Coordinates  (x4,y4,z4)  are  sensor  coordinates.  They  are  related  to  (x3,y3,z3)  by  the  gimbal 
angles  G1  (inner)  and  G2  (outer)  showm  in  Fig.  6.  When  the  FSM  is  used,  sensor  coordinates  and  satellite 
coordinates  are  the  same.  Thus,  in  nominal  configuration,  the  (x4,y4,24)  axes  of  Fig.  7  coincide  with  the 
(x2,y2,z2)  axes  of  Fig.  1 . 

The  following  three  angles  specify  the  satellite’s  location;  it  is  assumed  that  latitude  =  longitude  = 
0  at  t  =  0. 

•  El  =  (satellite  revolution  rate)xiime  =  U,t  is  the  angular  position  of  the  satellite  in  its  orbit, 
measured  from  the  equator  crossing.  If  the  orbital  inclination  is  zero,  this  is  the  satellite’s  latiwde. 

•  E2  =  (Earth  rotation  rale)  X  time  =  Ud  is  the  longitude  of  the  equator  crossing  point  of  the  orbit. 
If  the  orbital  inclination  is  zero,  this  is  also  the  satellite’s  longitude. 

•  7  .specifies  die  inclination  of  the  orbit  (in  degrees).  It  is  actually  the  complement  of  the  orbital 
inclination,  which  is  the  angle  between  the  plane  of  the  orbit  and  the  equator.  Thus,  referring  to  Fig.  1, 
7  =  0  describes  a  polar  orbit;  for  7  =  10°  (for  example),  the  satellite  passes  east  of  the  North  Pole  (yes, 
we  know  it’s  really  .south,  but  you  get  the  idea!). 

The  rotational  relations  among  die  four  sets  of  coordinates  are  given  below  (translational  relations 
are  not  needed  directly).  The  sines  and  cosines  of  angles  are  identified  by  the  prefixes  "S”  and  "C,” 
respectively.  Thus,  for  example,  SEI  »Sin(El),  CEl  ■Cos(El).  Specification  of  clockwise  (CW)  or 
counterclockwise  (CCW)  rotations  about  an  axis  assume  that  the  center  of  rotation  is  viewed  from  the 
+  axis  direction.  Thus,  the  usual  mathematical  definition  of  a  positive  rotation  coincides  with  CCW.  As 
will  be  apparent,  the  same  symbol  (e.g.  El)  is  often  (not  always!)  used  to  denote  both  an  angle  of  rotation 
and  the  matrix  that  effects  the  rotational  tran.sformation.  Notations  of  CW  and  CCW  rotations  assume  that 
die  angle  in  question  is  a  positive  number  and  that  other  angle.s  ii.  die  relevant  transformation  set  are  zero. 
For  example,  El  is  a  CCW  rotation  around  yl  only  if  E2  =  7  =  0;  otherwise  it  is  a  rotation  around 
some  other  axis  not  shown  in  Fig.  1. 
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The  transformation  from  EarJi  to  orbiting  coordinates  is  effected  as  follows; 


where 


El  = 


'x2' 

xl 

y2 

=  El  E7  E2 

yl 

z2 

zl 

CEl 

0  SEl] 

1 

1 

0  1  0 
-SEI  0  CEl 


CW  about  yl, 


(2) 


(3) 


and 


Ey 


E2  = 


1  0  0 
0  cy  -sy 
0  S7  cy 


CEl  SE2  0 
-SE2  CE2  0 
0  0  1 


CW  about  xl. 


CCW  about  zl. 


(4) 


(5) 


As  noted  above,  if  7  =  0,  then  E7  is  the  identity  matrix,  and  (El,  E2)  become  (latitude,  longitude)  of 
the  satellite’s  position.  Naturally,  the  inverse  transform  is  effected  by 


(6) 


"xT 

■x2‘ 

yl 

zl 

=  E2^  E7T  Er 

y2 

z2 

where  the  superscripted  T  denotes  the  transpose  matrix.  The  transformation  from  orbiting  to  satellite 
coordinate.s  is  given  by 


where 


and 


x3 

y3 

=  R 

z3 

fCR  - 

SR  0 

R  = 

SR  CR  0 

[0 

0  1 

CP  0 

-SP  ' 

P  - 

0 

1 

0 

SP  0 

CP 

ri 

0 

0 

Y  = 

C  CY 

-SY 

hi 

SY  CY 

x2 

y2 

z2 


CW  about  z2. 


CCW  about  y2. 


CW  about  x2. 


(7) 


(8) 

(9) 


(10) 


The  transformation  from  satellite  to  sensor  coordinates  is  effected  by  rotating  through  the  gimbal 
angles  shown  in  F'ig.  6: 
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x4  x3 

y4  =  G1  G2  y3 
z4  z3 


where 


G1  = 


CGI  0  -SGI 
0  1  0 
SGI  0  CGI 


CCW  about  y3 


CG2  SG2 
G2  =  -SG2  CG2 
0  0 


32  0 
32  0 
0  1 


CCW  about  z3. 


At  nominal  configuration,  G1  =  G2  =  0,  the  optic  axis  looks  straight  out  the  bottom  of  the  satellite. 
Positive  G1  gives  a  forward  look  (CCW  about  y3  axis);  positive  G2  gives  a  leftward  look  (CCW  about 
z3  axis). 

The  linear  array  is  in  the  (y4,z4)  plane,  looking  in  the  -y4  direction,  as  shown  in  Fig.  7.  The  scan 
mirror  directs  the  LOS  into  the  -x4  direction  and  scans  in  the  (x4,y4)  plane.  If  (in  nominal 
configuration)  one  sits  on  the  satellite,  facing  forward  (the  +z  direction  is  forward),  and  looks  down, 
scanning  is  done  from  left  to  right,  as  indicated  in  Fig.  1. 

Heretofore,  the  LOS  has  referred  to  the  center  of  the  sensor’s  FOV.  Other  look  directions  within  the 
FOV  result  from  specifying  the  cross-scan  location  of  an  arbitrary  detector  in  the  linear  array  by  the  angle 
El  (“elevation,”  not  to  be  confused  with  El  =  latitude)  and  the  in-scan  angular  position  specified  by  Az 
(azimuth).  At  the  center  of  the  scene,  El  =  Az  =  0.  The  components  of  the  resulting  look  vector  are 

(vx4,vy4,yz4)  =  (-CE1  •  CAz,  -CEl  •  SAz,SEl).  (1'^) 

Orienting  the  gimbal  to  point  at  a  specified  spot  on  the  ground,  or  finding  where  a  particular  LOS 
intersects  the  Earth’s  surface,  is  done  by  solving  the  vector  equation  Rj  +  v  =  Rj  (see  Fig.  1),  where 
R,  is  the  position  of  the  satellite  measured  from  the  center  of  the  Earth,  Rj  is  the  position  of  the 
observation  point  on  the  Earth,  and  v  is  the  vector  connecting  these  two  points.  Note  that  v=s^,  where 
s  is  the  distance  from  the  satellite  to  the  Earth  observation  point,  and  ^  is  the  LOS  look  direction  given 
in  Eq.  (14). 

To  begin  the  observation,  the  gimbals  must  be  oriented  to  direct  the  LOS  to  the  desired  observation 
point.  With  R,  and  Rj  known,  the  vector  between  them  is  immediately  found  from  v  =  Ra  -  Ri,  and  v 
is  normalized  to  the  look  direction  from  the  satellite.  The  solution  for  v  is  carried  out  in  Earth 
coordinates  (xl,yl,zl)  because  both  the  observed  point  on  the  surface  of  the  Earth  and  the  location  of 
the  satellite  are  specified  in  these  coordinates,  v  is  then  transformed  to  satellite  coordinates,  where  its 

components  are  (vx3,vy3,vz3),  and  the  gimbal  angles  required  to  point  the  center  of  the  FOV  in  this 
direction  are  given  by 

(vx3,vy3,vz3)  =  (CGI  •  CG2,CG1  •  SG2,SG1)  (15) 


G1  =  arcsin(vz3) 

G2  =  arctan(vy3/vx3). 
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3.1  Orientation  of  Gimbai  in  Satellite 

Figure  6  is  drawn  for  P'^minal  configuration,  that  is,  R  =  P  =  Y  =  0,  so  that  satellite  coordinates 
(x3,y3,z3)  are  the  same  as  orbiting  coordinates  (x2,y2,z2).  When  Gi  =  G2  =  0,  the  LOS,  after  the  scan 
mirror,  points  straight  down,  i.e.,  in  the  -112  direction. 

3.2  Orientation  of  Satellite  for  the  Fast  Steering  Mirror  (FSM) 

When  the  FSM  is  used,  the  optical  layout  is  as  shown  in  Fig.  7.  but  with  the  two-axis  FSM  in  place 
of  the  one-axis  scan  mirror  and  the  FPA-telescope  assembly  replaced  by  an  FPA-telescope-scan  mirror 
assembly.  For  an  off-nadir  observation,  the  satellite  must  be  oriented  so  that  the  LOS  is  directed  to  the 
desired  ground  point.  When  the  FSM  is  in  its  rest  position,  the  LOS  points  straight  out  the  bottom  of  the 
satellite.  Thus,  orienting  the  satellite  requires  a  pitch-roll  maneuver  to  point  its  bottom  at  the  desired 
ground  point.  (Yaw  could  be  used  but  is  .lut:  yaw  =  0  always.)  Tnen,  as  time  goes  on,  the  mirror  steers 
tlie  LOS  to  track  the  fixed  ground  point;  the  satellite’s  orientation  is  not  changed  during  the  observ’ation 
(except  that,  as  always,  it  tracks  the  local  vertical).  Scanning  is  done  separately  by  the  scan  mirror, 
internal  to  the  telescope. 

The  satellite  attitude  solution  for  this  ca.se  is  done  similarly  to  that  for  the  gimbaled  sensor,  except 
that  sensor  coordinates  (x4,y4,z4)  are  never  used  because  the  sensor  is  fixed  in  the  satellite  (so  (x4,y4,z4) 
=  (x3,y3,z3)).  Instead,  the  required  look  direction  v  is  found  in  orbital  coordinates  with  components 
(vx2,vy2,vz2),  and  the  satellite  is  rotated  about  its  pitch  and  roll  axes  to  achieve  proper  orientation. 
Proper  orientation  means  that  the  center  of  the  FOV  (which  always  looks  straight  out  the  bottom  of  the 
satellite)  points  in  the  desired  direction.  The  relation  between  the  look  direction  (in  orbiting  coordinates) 
and  P  and  R  is 


(vx2,vy2,vz2)  =  (-CP  •  CR.CP  •  SR,SP), 
so  pitch  and  roll  are  found  from 

P  =  arcsin(vz2) 

R  =  arctan(-vy2/vx2). 

3. 2. 1  Mirror  Reflections 

LOS’s  from  the  array  are  bounced  off  the  scanning  mirror  and/or  the  external  tracking  mirror.  In 
either  case,  as  shown  in  Fig.  8,  the  mirror’s  orientation  is  de.scribed  by  a  normal  vector  ft.  Given  ft  and 
the  incident  LOS  ,  the  reflected  LOS  is 

=  v,  -  2(^,  •  ft)ft. 

Or,  given  the  two  LOS’s,  the  required  mirror  position  is 


3.2.2  Jitter 

LOS  jitter  is  implemented  for  the  gimbaled  telescope  by  adding  randomly  chosen  angles  to  Gl  and 
G2.  T1  ,  means,  of  course,  that  the  directions  the  LOS  moves  in  response  to  these  jitter  angles  are 
mutually  perpendicular,  but  tlie  jitter  angle  applied  to  G2,  say  6a,  appears  on  the  LOS  as  6a‘Cos(Gl) 
(G 1  is  assumed  to  be  zero  in  the  position  shown  in  Fig.  6).  This  should  be  kept  in  mind  if  large  G 1  's  are 
used.  Furthermore,  jitter  angles  are  generated  only  when  the  block  ground  points  are  found  (cf.  Section 
4).  This  means  that  only  nine  (at  most)  independent  pairs  of  jitter  angles  are  used;  tiiey  are  applied  every 
64  samples  in  the  scan  direction. 


(19) 

(20) 
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MIRROR  NORMAL 

LOS  ; 

.  V,  i  V.  ^  TO  EARTH 

OPTICS  '  '  ' 


Fig.  8  —  The  three  unit  vectors  shown  represent  the 
mirror  normal  and  the  directions  of  the  incident  and 
reflected  beams.  If  any  two  of  these  arc  luiown,  the 
third  can  be  found  from  Eq.  (19)  or  Eq.  (20). 


For  the  FSM,  jitter  is  implemented  by  apply  the  randomly  chosen  angles  to 
of  the  satellite.  Thus,  both  appear  (essentially)  equally  on  the  LOS  because  it 
straight  out  the  bottom  of  the  satellite,  perpendicular  to  the  pitch  and  roll 
implementation  is  the  same  as  for  the  giinbaled  telescope. 

4.  CONSTRUCTION  OF  THE  GROUND  FOOTPRINT  (ITEM  5) 

Once  the  center  of  the  FOV  is  properly  directed,  the  line  of  sight  from  any  detector  in  the  focal  plane 
at  an  arbitrary  scan  mirror  angle  (i.e.,  the  LOS  to  an  arbitrary  location  in  the  FOV),  is  given  by  Eq.  (14) 
as  a  unit  vector  in  sensor  coordinates  (x4,y4,z4).  This  unit  vector  is  transformed  to  (xl,yl,zl) 
coordinates.  The  point  at  which  this  particular  LOS  intersects  the  Earth’s  surface  R2  must  now  be  found 
from  knowledge  of  R,  and  This  is  done  by  writing  v  =  sv,  noting  that  lRj|^  =  (R^  =  Earth 

radius),  ar:d.  solving  |R,  ♦  s('|^  =  R/  for  s: 

s  =  -R,  •  ❖  -  {(R,  •  ^)-  -  (Rf  -  R.,)}''"-  (21) 

(Note  that  R,  •  ^  is  a  negative  number  because  the  vectors  point  in  roughly  opposite  directions.)  Finally, 
R,  =  R,  +  sv  gives  the  ground  intersection  point. 

LOS  ground  intersection  points  are  calculated  for  (corners  oO  every  64th  pixel  in  the  linear  optica! 
array,  for  every  64th  sample  in  the  scan  direction.  That  is,  ground  points  are  calculated  for  the  corners 
of  blocks  that  will  later  be  subdivided  into  4096  (64  x  64)  pixel  locations,  one  for  each  of  the  4096 
samples  in  that  block  of  the  image.  The  relation  between  pixels  and  samples  is  given  by  the  oversampling 
factors  in  the  scan  (X)  and  cross-scan  (Y)  directions,  denoted  by  OSFX  and  OSFY.  Thus,  if  OSFX  = 
2,  two  samples  per  pixel-length  are  in  the  scan  direction  (i.e.,  two  samples  per  dwell).  With  left-to-right 
.scanning,  this  means  the  total  distance  between  the  left  edge  of  a  pixel  on  the  first  sample  and  its  right 
edge  on  the  64th  sample  is  (1  +63/QSFX)  x  d,,  where  d,  is  the  horizontal  length  of  a  pixel.  Similarly, 
the  distance  between  the  top  of  the  first  pixel  and  the  bottom  of  the  64th  is  (H-63/OFSY)  x  d,.  where 
d,  is  the  vertical  length  of  a  pixel.  Horizontal  and  vertical  lengths  are  equal  in  the  focal  plane,  but 
(usually)  not  when  projected  on  the  ground. 

For  the  maximum  image  size  of  512  x5 12  samples  (which  can  cover  a  considerably  smaller  ground 
area  than  the  whole  FOV  of  512  x  512  pixels),  LOS  ground  intersection  points  are  contained  in  two  data 
arrays  giving  the  latitude  and  longitude  of  256  points  that  divide  the  ground  footprint  into  64  separate 
blocks.  These  arrays  are  called  X(8,8).C(4)  and  Y(8,8).C(4);  the  (8,8)  indices  label  the  64  blocks  and 
the  (4)  index  labels  the  corners  of  each  block.  For  the  usual  image  size  of  64x256,  only  four  of  these 
blocks  are  used.  These  blocks  are  then  operated  on  individually  by  the  rest  of  the  program. 


the  pitch  and  roll  axes 
always  looks  (nearly) 
axes.  Otherwise,  the 
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The  next  step  is  to  convert  latitudes  and  longitudes  into  “HR  pixels,”  that  is,  units  in  which  the  unit 
of  length  is  the  size  of  a  pixel  in  the  background  HR  data  set,  and  the  origin  is  tlie  fixed  stare  point,  i.e., 
the  center  of  the  512x512  pixel  FOV. 

The  scan  mirror  executes  a  complete  scan-and-flyback  cycle  in  a  frame  period  time,  usually  0.5  s. 
Its  duty  cycle  is  80%;  the  scan  motion  takes  0.4  s,  the  flyback  takes  0.1  s.  The  time  required  to  scan 
through  tlie  usual  horizontal  range  of  256  samples  is  therefore  0.4  x (2.56/5 12)/OSFX,  where  OSFX  is 
the  oversampling  factor  in  the  scan  direction.  The  satellite’s  position  is  updated  and  new  gimbal  angles 
(or  FSM  orienution  angles)  are  calculated  for  each  scan  position,  that  is,  for  the  vertical  edges  of  the 
64-sample-long  blocks. 

Tlte  edges  of  these  blocks  are  then  divided  into  64  equal  increments,  which  overlap  by  OSFX  and 
OSFY  in  the  scan  and  cross-scan  directions,  re.spectively.  The  interior  of  each  block  is  then  gridded  into 
64  x  64  overlapping  pixel  locations,  one  for  each  sample.  The  pixel  footprints  within  a  block  are  all 
equally  spaced.  This  means  that  the  change  in  size  of  the  pixel  footprints,  which  should  take  place 
gradually  over  the  FOV,  instead  happens  in  jumps  at  the  block  boundaries.  Except,  perhaps,  very  near 
the  horizon,  where  severe  footprint  distortions  take  place,  this  approximation  will  not  significantly  affect 
imagery. 

S.  OPTICAL  BLUR  (ITEM  7) 

The  optical  blur  function  is  chosen  to  simulate  matched  optics,  which  is  precisely  defined  to  mean 
that  the  diameter  of  the  first  zero-intensity  circle  of  the  theoretical  diffraction  pattern  for  an  unobscured 
circular  aperture  is  equal  to  the  width  of  the  detector  (which  is  square).  Since  reality  will  not  match  this 
precise  definition,  the  optical  blur  function  is  taken  to  be  Gaussian.  The  standard  deviation  of  the 
Gaussian  is  chosen  so  that  70%  (as  opposed  to  the  theoretical  value  of  85%)  of  the  light  falls  within  a 
detector  when  the  blur  spot  is  centered  on  it.  In  detector-width  units,  this  is  cr  =  0.36. 

Referring  to  Fig.  9,  we  see  that  the  contribution  to  a  detector  from  an  HR  pixel  with  unit  intensity 
located  distances  u,v  from  the  detector’s  center  is 


/  -y*|/2  e  'v*i/2 

D(u,v)  =  I  B(x,y)dxdy, 

J  y-1/2  j  -v-l/2 


wheie 


(22) 


B(x,y) 


1 

2n(f^ 


exp 


x’  +  y 
~2?' 


2  1 


(23) 


is  the  assumed  optical  blur  function,  which  is  Gaussian  (with  the  same  a)  in  both  its  arguments.  The 
shape  of  the  HR  pixel  (which,  as  shown  in  Fig.  will  usually  not  be  square)  does  not  enter  tJiis 
expression  -  it  is  sufficiently  smaller  than  the  blur  circle  (^l/6)  that  its  shape  makes  no  appreciable 
difference,  so  it  is  treated  as  a  point  source.  The  condition  that  70%  of  the  energy  in  the  blur  circle  falls 
within  a  detector  becomes  D(0,0)  =  0.7,  and  is  fulfilled  for  a  =  0.36. 

D(U,v)  can  be  expres.,ed  analytically  as  a  combination  of  products  of  error  functions.  It  can  be 
expressed  usefully,  to  a  very  good  approximation,  as  the  same  Gaussian  form  as  B(x,y),  but  with  Ou  - 
0.50. 
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Fig  9  —  Only  those  high-resolution  pixels  whose  centers 
lie  within  1.3  detector-widths  of  the  center  of  a  particular 
detector  contribute  to  that  detector's  output. 


The  optical  blur  caused  by  the  scanning  motion  can  be  taken  into  account  by  noting  that  the  motion 
is  in  the  u  direction  only  (in  the  focal  plane  coordinates  of  Fig.  9)  and  that  the  image  on  the  focal  plane 
moves  a  distance  1/OSFX,  where  unity  is  the  width  of  a  detector.  Thus,  the  contribution  of  an  HR  pixel 
becomes  the  average  of  D(u,v)  over  the  distance  the  image  moves; 

/  u*I/?  •OSI-X»  /'\A\ 

C(u,v)  =  OSFX  I  D(w,v)dw.  (24) 

Ju-i.'(:*osFXi 

Look-up  tables  are  calculated  for  the  function  C(u,v)  and  its  derivatives.  Note  that  C(u,v)  is 
separable;  C(u,v)  =  C,(u)*Cy(v).  Thus,  for  arbitrary  u,v, 


C(u,v)  =  [C,(u,)  ♦  (u-u,)«  C.(u.)l  »  lC^(vp  (v-vp*C/(vp]  (25) 

where  i  and  j  denote  values  stored  in  the  tables,  and  '  denotes  derivative.  That  is,  C(u,v)  is  given  by  a 
two-term  Taylor  series  for  both  its  arguments.  By  using  this  approach,  values  for  C(u,v)  as  a  continuous 
function  can  be  found  without  doing  the  integral  in  Eq.  (22)  each  time. 

6.  INTEGRATE  OVER  BACKGROUND  SCENE  (ITE.M  8) 

At  this  point,  the  ground  footprint  of  every  detector  is  known  for  every  sample  that  will  appear  in 
the  simulated  scene.  For  each  sample,  die  program  must  now  locate  those  HR  pixels  that  will  contribute 
significantly  to  that  sample  and  calculate  their  contributions. 

6.1  Ground-Point  Fold-In 

A  basic  problem  with  ground  truth  data  is  that  most  existing  databases  with  acceptable  resolution 
come  from  aircraft-borne  instruments,  hence  do  nut  cover  as  large  an  area  as  a  satellite-borne  sensor. 
Typically,  as  die  airplane  flies,  it  generates  data  covering  a  scene  that  is  long  in  one  direction  but  short 
in  liie  other  (although  even  the  long  dimension  is  usually  mit  long  enough,  rarely  being  more  than  10  to 
20  km).  These  scenes  must  be  used  to  tile  a  larger  region  for  our  satellite  simulations.  This  is  done, 
conceptually,  by  “folding  out”  the  aircraft  data  so  that  the  resulting  large  scene  does  not  have 
discontinuities  at  the  edges  of  the  original  small  scene  In  actuality,  this  folding  out  of  the  HR  data  is 
done  by  separately  “folding  in”  the  x-  and  y-indexes  of  particular  points  in  the  simulated  scene,  as  shown 
schematically  in  Fig.  10. 
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Fig.  10  —  A,  B,  and  C  represent  arbitrary  points  in  the  simulated  scene.  They  arc  folded  into  the  region 
covered  by  the  high-resolution  data  in  the  manner  shown. 


To  see  how  this  is  done,  we  consider  the  background  scene  to  cover  an  interval  [0,N]  along  the  x 
axis  with  pixels  labeled  n,  1  ^  n  ^  N  (the  origin  is  the  center  of  the  sensor’s  512x512  pixel  FOV).  An 
arbitrary  positive  point  x  can  be  specified  by  x  =  M.d,  where  M  and  d  are  its  integer  and  decimal  parts, 
respectively.  We  want  to  find  the  new  point  x',  0  ^  x"N,  into  which  x  folds.  If  0  M  :S  N,  then, 
trivially,  x'  =  x.  But  if  M  iS:  N,  we  must  first  evaluate  K  =  T(M/N),  using  the  computer’s 
real-to-integer  truncation  function,  T(±integer.decimal)=±integer.  If  K  is  odd,  then  the  new  point  is 
x'  =  (K+  l)N-x  (the  interval  containing  x  will  be  reversed  when  it  is  folded  into  the  basic  [0,N]  interval). 
If  K  is  even,  the  new  point  is  x'  =  x-KN.  For  example,  taking  N  =  100  and  x  =  127.3,  we  find  that 
K  =  1  and  x'  =  72.7,  which  is  correct.  Similarly,  if  x  =  227.3,  then  K  =  2  and  x'  =  27.3. 

For  an  arbitrary  negative  point,  x=-M.d,  the  appropriate  formulas  are  x'  =  x  +  (K+  1)N  for  K  odd, 
and  x'  =  — KN— X  for  K  even.  In  this  case,  the  reversal  of  the  interval  occurs  for  K  even.  Thus,  for  x  = 
—27.3,  K  =  0  (which  is  even)  and  x'  =  27.3.  For  x  =  —127.3,  K  =  1  and  x'  =  72.7. 

6.2  Location  of  HR  Pixels  Relative  to  Center  of  Detector 

The  location  of  HR  pixels  must  be  found  in  focal  plane  coordinates.  Up  to  now,  the  program  has 
located  focal  plane  coordinates  on  the  ground  in  units  of  HR  pixels.  This  requires  a  transformation 
between  the  sets  of  coordinates  shown  in  Fig.  11,  where  i  and  ]  are  perpendicular  unit  vectors  in  the 
plane  of  HR  pixels  and  and  <>2  vectors  (specifying  the  edges  of  detectors)  that  are 

perpendicular  on  the  focal  plane. 

Angles  0,,  $2  are  defined  as  shown  in  Fig.  11,  so  that 

=  COS0J  I  +  sin0,  J 
and 

^2  =  COS02  ‘  +  sin02  J  •  (26) 
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Fig.  1 1  —  TTiis  figure  shows  (schcmaiically)  the  same  clemenU 
shown  in  Fig.  9,  but  projected  on  the  ground  instead  of  projected  on 
the  focal  plane. 


are  not  perpendicular,  but  they  span  the  x,y  plane,  so  an  arbitrary  HR  pixel  at  xi  +  yj  can  also 
be  described  as  having  the  location  a<>,  +  bC-,,  where  a  an  !  b  are  found  by  solving  a^,  +  b^j  =  xi  +  yj 
to  get 


xsin^j  -  ycos0, 
sin((9j  -  0,) 


(27) 


xsind,  -  ycosfi, 
sin(0,  -  0,) 


(28) 


Because  of  the  Earth’s  curvature,  a  simulated  pixel  projected  on  the  ground  is  slightly  different  from 
a  parallelogram,  so  opposite  sides  are  averaged  to  find  and 

The  last  step  to  obtain  the  focal  plane  position  of  an  HR  pixel  is  to  divide  a  and  b  (from  Eqs.  (27) 
and  (28)),  which  are  in  units  of  HR  pixels,  by  the  magnitudes  of  and  to  convert  them  to  focal  plane 
units.  The  resulting  numbers  (u,v)  can  now  be  checked  to  see  if  they  are  within  the  circle  of  radius  1.3 
that  will  make  a  significant  contribution  to  the  detector  in  question. 

6.3  Sum  Up  the  Contributions  of  Nearby  HR  Pixels 

The  contributions  from  all  relevant  HR  pixels  must  now  be  added  up  to  find  the  signal  on  the 
defector  Tbe..se  contributions  are  determined  by  the  location  of  each  HR  pixel  in  relation  to  the  location 
of  the  detector  ground  footprint  for  the  sample  under  consideration.  The  necessary  values  are  found  from 
the  look-up  tables  generated  for  C(u,v),  explained  in  Section  5. 

In  principle,  of  course,  C(u,v)?i0  no  matter  how  large  u,v  are,  so  all  HR  pixels  should  be  summed 
over.  To  save  computing  time,  a  cut-off  distance  of  1 .3  is  imposed,  as  indicated  in  Fig.  9;  only  HR  pixels 
whose  centers  lie  inside  die  circular  region  (-1.3  ^  (u‘  +  v-)l/2  ^  1.3)  are  summed  over.  For  a  = 
0.36  (oc  =  0.50),  this  includes  about  97%  of  the  area  under  the  function  C(u,v),  but  only  about  90%  of 
the  power  in  a  real,  nearly  diffraction-limited,  blur  function.  Thus,  about  10%  of  the  optical  power  is 
not  handled  correctly,  and  the  influence  of  an  extremely  bright  region  will  not  be  felt  far  and  wide  across 
the  focal  plane,  as  in  reality  it  would. 
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Since  the  number  of  HR  pixels  that  happen  to  fall  inside  the  acceptable  region  is  variable,  the  sum 
of  their  contributions  is  normalized  as  follows: 

52  Signal(m)  »  Blur(m) 

m*l ..  .pixels  (29) 

52  Blur(m) 

m*l . .  .n  ptxi^ls 


where  Biur(m)  is  the  value  of  C(u,v)  for  the  HR  pixel  labeled  m.  The  range  of  1.3  is  chosen  to  give 
extremely  accurate  simulations,  having  a  precision  of  about  0.01  /iF  for  input  values  around  500  to 
1000  This  is  far  below  the  usual  noise  level  of  0.3  ^F.  Thus,  while  extreme  precision  may  be  desired 
for  some  purposes  (exploring  the  ultimate  power  of  algorithms,  for  example),  computing  time  can  be 
reduced  for  ordinary  applications  by  reducing  the  range  to  about  1. 

7.  REGISTRATION  AND  RESAMPLING  OTEM  II) 

To  prepare  the  data  for  an  FDSP,  successive  images  must  be  accurately  registered  and  resampled. 
ITe  first  step  in  the  process  is  to  determine  the  displacement  between  images. 

7.1  Displacement  Estimation 

Because  different  parts  of  the  scene  move  by  different  amounts,  the  scene  is  broken  up  into 
32  X  32-sample  blocks,  over  which  displacement  is  acceptably  uniform.  The  measured  displacement  is  then 
used  to  resample  the  central  16 x  16  subblock,  and  another  32x32  block  is  done.  Over  this  small  region, 
disphacement  differs  insignificantly  from  a  translation.  Displacement  is  found  by  assuming  the  satellite 
has  a  navigation  and  orientation  system  (composed  of  star  trackers  and  precision  gyroscopes)  capable  of 
determining  (to  within  10  to  20  /trad)  where  each  pixel  is  located  on  the  ground.  The  residual  10  to 
20  /(tad  misregistration  can  then  be  found  by  applying  a  gradient  technique  to  the  radiance  differences 
of  the  two  scenes,  or  by  apply  ing  a  Fourier  technique  (5);  the  former  is  the  more  accurate  of  the  two, 
as  currently  implemented.  Even  greatei  accuracy  can  be  obtained  by  implementing  a  direct  search  that 
minimizes  the  clutter  in  the  resulting  difference  frame  |4j.  The  computer  keeps  track  of  exact  ground 
positions  for  all  images,  so  the  accuracy  of  various  resampling  methods  can  be  readily  assessed. 

7.2  Resampling  the  Images 

Oni.e  the  displacement  between  two  images  has  been  determined,  they  must  be  resampled  on  a 
common  grid.  Currently,  this  is  done  with  filtering  interpolators  |6),  which  achieve  excellent  clutter 
suppression  by  applying  a  spectral  filter  to  the  un.shifted  frame  that  matches  the  filtering  effect  of  applying 
the  interpolator  to  ihe  frame  that  is  shifted.  The  result,  for  most  cloud-free  scenes,  is  clutter  reduction 
below  the  level  of  sensor  noise.  Usually,  only  two  images  are  compared  (that  is,  frame  differencing  is 
not  done  to  a  higher  older  than  one);  consequently,  .some  improvement  can  be  realized  by  implementing 
Dual  Difference  Filters,  which  constitute  the  optimal  approach  to  this  problem  |7]. 

8.  IMAGE  DIAGNOSTICS 

EESP  contains  some  options  which  do  not  contribute  directly  to  the  generation  of  images  or  the 
detection  of  targets,  but  can  assist  the  user  in  image  analysis. 

Whole-image  histograms  can  be  generated  and.br  line  plots  made  of  individual  horizontal  or  vertical 
slices  through  the  scene,  these  features  are  standard  and  require  no  special  e.xplanation.  Also  provided 
is  an  option  that  calculates  and  displays  the  standard  deviations  of  32  x32  subblocks  of  the  image.  Each 
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Standard  deviation  is  calculated  using  the  subblock  mean,  so  the  result  is  an  indication  of  which  parts  of 
the  scene  contain  the  most  variable  structure  hence  should  be  the  strongest  sources  of  clutter. 

But  better  measures  of  clutter  are  a'. ..  le.  The  Cumulative  Statistics  option  attempts  to  give  a  clear 
idea  of  the  clutter  content  of  a  scene  by  using  as  few  numbers  as  possible.  It  lists  the  maximum, 
minimum,  mean,  and  standard  deviation  (a)  of  an  image.  It  also  gives  A.  and  A^,  which  are  the 
root-mean-square  values  of  the  difference  between  adjacent  samples.  The  defining  equations  for  these 
quantities  are 


-i.  ■  jE  ‘h,..  -  j"  (30) 

and 

^-jE(^.-..,  -^,)j".  (31) 

where  s.^  denotes  the  value  of  sample  (i,j)  and  the  sum  is  over  all  samples  in  the  image,  with  the  obvious 
exception  of  those  samples  at  the  edge  for  which  i  -P  1  or  J  -t-  1  falls  outside  the  image,  (lliis  edge  effect 
applies  to  many  of  the  image  processing  algorithms  used  by  EESP,  especially  resampling,  and  is  for 
this  reason  that  some  displayed  images,  .such  as  Fig.  4,  have  the  outermost  eight  samples  bit  like  J  cut  or 
replaced  by  the  scene  average.)  The  reason  why  A,  and  A^  are  given  separately  is  that  images  need  not 
be  isotropic,  and  certainly  won't  be  when  the  sampling  rates  in  the  x  and  y  directions  are  different.  Note 
that  J  labels  the  x,  or  horizontal,  direction,  i  labels  the  y,  or  vertical,  direction  (i  =  j  =  1  at  the  upper 
left-hand  corner  of  the  image). 

A  has  been  found  to  be  a  belter  indicator  of  scene  cluttei  than  a  because  it  is  a  local  measure  of 
variability.  A  scene  that  is  substantially  brighter  in  one  region  than  another  can  have  a  large  value  for 
a  because  it  exhibits  large  variations  from  the  mean,  but,  if  the  radiance  grades  smoothly  from  one  region 
to  the  other,  it  will  not  present  a  real  challenge  to  a  Frame  Differencing  Signal  Processor  (FDSP).  Clutter 
leakage  through  an  FDSP  depends  exclusively  on  the  local  properties  of  the  image.  Sharp  edges,  for 
example,  often  “leak  through’’  an  FDSP,  producing  artifacts  in  the  differenced  image.  (However,  our 
work  has  shown  that,  given  reasonable  sampling  rates  and  good  registration  and  resampling  algorithms, 
even  the  clutter  leakage  from  sharp  edges  can  be  reduced  to  below  the  level  of  sensor  noise.) 

The  A’s  are,  of  course,  closely  related  to  the  radiance  gradients  in  the  scene,  if  R(x,y)  represents  the 
scene  radiance  and  G,  =  dR/dx,  then  A,  =  Gb'dx,  where  dx  is  the  separati.jn  between  samples  in  the 
X  direction.  Similarly,  A^  =  Gj  "dy.  Thus,  A  is  an  rms  measure  of  the  local  rate  of  change  in  a  scene, 
and  this  is  a  much  more  important  quantity  in  determining  its  clutter  characteristics  than  is  a. 

9.  CONCLUSION  AND  RETROSPECTIVE 

The  major  features  of  the  End-to-End  Simulation  Package  (EESP)  have  been  described.  It  is  a 
versatile,  easily  modified  means  of  generating  accurately  simulated  imagery  and  of  assessing  signal 
proces.sing  algorithms.  In  another  report,  EESP  has  been  used  to  demon.strate  the  feasibility  of  IR 
detection  of  bomber-sized  airborne  targets  from  space-borne  platforms,  w  iifi  high  reliability  and  very  low 
false  alarm  rates  14,8]. 

The  software  package  described  here  is  the  culmination  of  an  intensive,  five-year  effort  bv  the 
Advanced  Concepts  Branch  per.sonnel.  Its  earliest  roots  trace  back  to  work  done  a  decade  ago  at  NRL 
to  a.sses.s  the  pro.spective  usefulness  of  the  Defen.sc  Advanced  Research  Projects  Agency  (DARP.A) 
proposed  demonstration  ot  IR  detection  from  space,  called  Teal  Ruby.  An  important  lesson  of  this  work 
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is  that  the  analyst  and  the  programmer  must  each  be  aware  of  the  other’s  problems.  Early  failures  to 
observe  this  rule  resulted  in  problems  which,  when  corrected,  cut  the  running  time  needed  for  a  VAX 
750  to  generate  a  nadir-looking  frame  from  two  hours  to  fifteen  minutes! 

The  first  author  has  been  involved  in  this  effort  from  the  beginning  and  is  acutely  awaie  of  the 
tremendous  amount  of  analytical  and  programming  work  that  has  been  expended  to  achieve  the  current 
level  of  capability.  But  when  all  is  said  and  done,  1  am  reminded  of  the  immortal  words  of  Omar 
Khayyam: 

Op  from  Earth’s  center  through  the  seventh  gate, 

I  arose  and  on  the  throne  of  Saturn  sate. 

And  many  a  knot  unravelled  on  the  road. 

But  not  the  ma.ster-knot  of  human  fate. 
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